{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ExprOptimization.jl\n",
    "\n",
    "ExprOptimization.jl is a Julia package for optimizing Julia expressions.  The package implements algorithms to optimize expression trees derived from a grammar to optimize a user-defined objective function.  The package depends on ExprRules.jl.\n",
    "\n",
    "## Installation\n",
    "\n",
    "To install the package:\n",
    "\n",
    "    Pkg.add(\"ExprOptimization\")\n",
    "\n",
    "## Usage\n",
    "\n",
    "To start using the package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "using ExprOptimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example -- Symbolic Regression\n",
    "\n",
    "We consider the example of finding an algebraic expression that approximates a given function.\n",
    "\n",
    "First, we define a grammar:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1: Real = x\n",
       "2: Real = Real * Real\n",
       "3: Real = Real + Real\n",
       "4: Real = Real - Real\n",
       "5: Real = 1.0\n",
       "6: Real = 2.0\n",
       "7: Real = 3.0\n",
       "8: Real = 4.0\n",
       "9: Real = 5.0\n"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grammar = @grammar begin\n",
    "    Real = x\n",
    "    Real = Real * Real\n",
    "    Real = Real + Real\n",
    "    Real = Real - Real\n",
    "    Real = |(1.0:5.0)\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define the ground truth expression and loss function by overloading the `loss` function in ExprOptimization.  The loss function returns the real-valued loss of a given expression tree.  The loss is minimized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loss (generic function with 1 method)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ground_truth(x) = x*x + 2x + 1\n",
    "function loss(tree::RuleNode, grammar::Grammar)\n",
    "    ex = get_executable(tree, grammar)\n",
    "    los = 0.0\n",
    "    global x\n",
    "    for x = -5.0:1.0:5.0\n",
    "        los += abs2(eval(ex) - ground_truth(x))\n",
    "    end\n",
    "    los\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once these are defined, we can use any of the implemented algorithms to perform the optimization.\n",
    "\n",
    "### Monte Carlo\n",
    "\n",
    "Monte Carlo (MC) draws a number of random expression trees from the grammar and returns the one with the best loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mM\u001b[22m\u001b[1mo\u001b[22m\u001b[1mn\u001b[22m\u001b[1mt\u001b[22m\u001b[1me\u001b[22m\u001b[1mC\u001b[22m\u001b[1ma\u001b[22m\u001b[1mr\u001b[22m\u001b[1ml\u001b[22m\u001b[1mo\u001b[22m \u001b[1mM\u001b[22m\u001b[1mo\u001b[22m\u001b[1mn\u001b[22m\u001b[1mt\u001b[22m\u001b[1me\u001b[22m\u001b[1mC\u001b[22m\u001b[1ma\u001b[22m\u001b[1mr\u001b[22m\u001b[1ml\u001b[22m\u001b[1mo\u001b[22ms\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "MonteCarlo\n",
       "```\n",
       "\n",
       "Monte Carlo.\n",
       "\n",
       "# Arguments:\n",
       "\n",
       "  * `num_samples::Int`: number of samples\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n"
      ],
      "text/plain": [
       "```\n",
       "MonteCarlo\n",
       "```\n",
       "\n",
       "Monte Carlo.\n",
       "\n",
       "# Arguments:\n",
       "\n",
       "  * `num_samples::Int`: number of samples\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?MonteCarlo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(:(x * (2.0 + ((5.0 - 5.0) + (x + (3.0 - 3.0)))) + (3.0 - 1.0 * 2.0)), 0.0)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srand(10)\n",
    "p = MonteCarlo(20000, 6)\n",
    "results_mc = optimize(p, grammar, :Real, loss)\n",
    "(results_mc.expr, results_mc.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"130.67pt\" height=\"182.518pt\" viewBox=\"0 0 130.67 182.518\" version=\"1.1\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-0\">\n",
       "<path style=\"stroke:none;\" d=\"\"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-1\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.84375 -3.046875 C 4.84375 -3.40625 4.5 -3.40625 4.375 -3.40625 L 2.953125 -3.40625 L 2.953125 -4.828125 C 2.953125 -4.9375 2.953125 -5.296875 2.609375 -5.296875 C 2.265625 -5.296875 2.265625 -4.9375 2.265625 -4.828125 L 2.265625 -3.40625 L 0.84375 -3.40625 C 0.734375 -3.40625 0.375 -3.40625 0.375 -3.046875 C 0.375 -2.703125 0.734375 -2.703125 0.84375 -2.703125 L 2.265625 -2.703125 L 2.265625 -1.28125 C 2.265625 -1.15625 2.265625 -0.8125 2.609375 -0.8125 C 2.953125 -0.8125 2.953125 -1.15625 2.953125 -1.28125 L 2.953125 -2.703125 L 4.375 -2.703125 C 4.5 -2.703125 4.84375 -2.703125 4.84375 -3.046875 Z M 4.84375 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-2\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.546875 -2.140625 C 4.546875 -2.296875 4.421875 -2.390625 4.40625 -2.40625 C 4.15625 -2.578125 3.484375 -2.90625 3.21875 -3.046875 L 4.296875 -3.625 C 4.421875 -3.703125 4.546875 -3.765625 4.546875 -3.953125 C 4.546875 -4 4.546875 -4.28125 4.140625 -4.28125 L 2.90625 -3.5625 C 2.9375 -3.828125 2.9375 -4.484375 2.9375 -4.78125 C 2.9375 -4.859375 2.9375 -5.1875 2.609375 -5.1875 C 2.28125 -5.1875 2.28125 -4.859375 2.28125 -4.78125 C 2.28125 -4.484375 2.296875 -3.828125 2.3125 -3.5625 L 1.234375 -4.203125 C 1.09375 -4.28125 1.078125 -4.28125 1 -4.28125 C 0.8125 -4.28125 0.671875 -4.109375 0.671875 -3.953125 C 0.671875 -3.765625 0.78125 -3.703125 0.921875 -3.640625 L 2 -3.046875 L 0.921875 -2.46875 C 0.8125 -2.390625 0.671875 -2.328125 0.671875 -2.140625 C 0.671875 -2.09375 0.671875 -1.796875 1.078125 -1.796875 L 2.3125 -2.515625 C 2.296875 -2.265625 2.28125 -1.609375 2.28125 -1.3125 C 2.28125 -1.21875 2.28125 -0.890625 2.609375 -0.890625 C 2.9375 -0.890625 2.9375 -1.21875 2.9375 -1.3125 C 2.9375 -1.609375 2.9375 -2.265625 2.90625 -2.515625 C 3.515625 -2.1875 3.15625 -2.375 3.84375 -1.96875 C 4.109375 -1.796875 4.140625 -1.796875 4.21875 -1.796875 C 4.421875 -1.796875 4.546875 -1.984375 4.546875 -2.140625 Z M 4.546875 -2.140625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-3\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.9375 -0.296875 C 4.9375 -0.609375 4.6875 -0.609375 4.53125 -0.609375 L 4.140625 -0.609375 L 2.859375 -2.21875 L 4 -3.6875 L 4.390625 -3.6875 C 4.53125 -3.6875 4.8125 -3.6875 4.8125 -3.984375 C 4.8125 -4.296875 4.546875 -4.296875 4.390625 -4.296875 L 3.234375 -4.296875 C 3.078125 -4.296875 2.828125 -4.296875 2.828125 -4 C 2.828125 -3.6875 3.046875 -3.6875 3.3125 -3.6875 L 2.578125 -2.6875 L 1.828125 -3.6875 C 2.078125 -3.6875 2.296875 -3.6875 2.296875 -4 C 2.296875 -4.296875 2.046875 -4.296875 1.90625 -4.296875 L 0.734375 -4.296875 C 0.59375 -4.296875 0.328125 -4.296875 0.328125 -3.984375 C 0.328125 -3.6875 0.59375 -3.6875 0.734375 -3.6875 L 1.140625 -3.6875 L 2.3125 -2.21875 L 1.078125 -0.609375 L 0.671875 -0.609375 C 0.53125 -0.609375 0.265625 -0.609375 0.265625 -0.296875 C 0.265625 0 0.53125 0 0.671875 0 L 1.84375 0 C 2 0 2.25 0 2.25 -0.296875 C 2.25 -0.609375 2.03125 -0.609375 1.71875 -0.609375 L 2.578125 -1.828125 L 3.46875 -0.609375 C 3.1875 -0.609375 2.96875 -0.609375 2.96875 -0.296875 C 2.96875 0 3.21875 0 3.375 0 L 4.53125 0 C 4.671875 0 4.9375 0 4.9375 -0.296875 Z M 4.9375 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-4\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.703125 -0.40625 L 4.703125 -0.671875 C 4.703125 -0.859375 4.703125 -1.078125 4.359375 -1.078125 C 4.015625 -1.078125 4.015625 -0.890625 4.015625 -0.609375 L 1.640625 -0.609375 C 2.234375 -1.109375 3.1875 -1.859375 3.625 -2.265625 C 4.25 -2.828125 4.703125 -3.453125 4.703125 -4.25 C 4.703125 -5.453125 3.703125 -6.203125 2.484375 -6.203125 C 1.3125 -6.203125 0.515625 -5.390625 0.515625 -4.53125 C 0.515625 -4.171875 0.796875 -4.0625 0.96875 -4.0625 C 1.171875 -4.0625 1.40625 -4.234375 1.40625 -4.5 C 1.40625 -4.625 1.359375 -4.75 1.265625 -4.828125 C 1.421875 -5.28125 1.890625 -5.59375 2.4375 -5.59375 C 3.25 -5.59375 4.015625 -5.140625 4.015625 -4.25 C 4.015625 -3.5625 3.53125 -2.984375 2.875 -2.4375 L 0.671875 -0.578125 C 0.578125 -0.5 0.515625 -0.453125 0.515625 -0.3125 C 0.515625 0 0.765625 0 0.921875 0 L 4.3125 0 C 4.640625 0 4.703125 -0.09375 4.703125 -0.40625 Z M 4.703125 -0.40625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-5\">\n",
       "<path style=\"stroke:none;\" d=\"M 3.234375 -0.625 C 3.234375 -0.984375 2.9375 -1.25 2.625 -1.25 C 2.25 -1.25 2 -0.9375 2 -0.625 C 2 -0.265625 2.296875 0 2.609375 0 C 2.984375 0 3.234375 -0.3125 3.234375 -0.625 Z M 3.234375 -0.625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-6\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.71875 -3.046875 C 4.71875 -4.890625 3.703125 -6.203125 2.609375 -6.203125 C 1.5 -6.203125 0.5 -4.859375 0.5 -3.046875 C 0.5 -1.203125 1.515625 0.109375 2.609375 0.109375 C 3.734375 0.109375 4.71875 -1.21875 4.71875 -3.046875 Z M 4.03125 -3.15625 C 4.03125 -1.671875 3.390625 -0.5 2.609375 -0.5 C 1.828125 -0.5 1.1875 -1.671875 1.1875 -3.15625 C 1.1875 -4.609375 1.875 -5.59375 2.609375 -5.59375 C 3.34375 -5.59375 4.03125 -4.609375 4.03125 -3.15625 Z M 4.03125 -3.15625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-7\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.671875 -3.046875 C 4.671875 -3.40625 4.3125 -3.40625 4.203125 -3.40625 L 1.03125 -3.40625 C 0.90625 -3.40625 0.5625 -3.40625 0.5625 -3.046875 C 0.5625 -2.703125 0.90625 -2.703125 1.03125 -2.703125 L 4.203125 -2.703125 C 4.3125 -2.703125 4.671875 -2.703125 4.671875 -3.046875 Z M 4.671875 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-8\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.703125 -1.890625 C 4.703125 -2.90625 3.9375 -3.875 2.765625 -3.875 C 2.40625 -3.875 1.96875 -3.8125 1.5625 -3.59375 L 1.5625 -5.484375 L 3.953125 -5.484375 C 4.109375 -5.484375 4.359375 -5.484375 4.359375 -5.78125 C 4.359375 -6.09375 4.109375 -6.09375 3.953125 -6.09375 L 1.28125 -6.09375 C 0.953125 -6.09375 0.875 -6 0.875 -5.6875 L 0.875 -3.03125 C 0.875 -2.84375 0.875 -2.625 1.203125 -2.625 C 1.375 -2.625 1.421875 -2.671875 1.5 -2.765625 C 1.765625 -3.09375 2.171875 -3.265625 2.75 -3.265625 C 3.5625 -3.265625 4.015625 -2.546875 4.015625 -1.890625 C 4.015625 -1.09375 3.3125 -0.5 2.46875 -0.5 C 2.1875 -0.5 1.5625 -0.578125 1.28125 -1.125 C 1.328125 -1.171875 1.40625 -1.25 1.40625 -1.453125 C 1.40625 -1.734375 1.171875 -1.890625 0.96875 -1.890625 C 0.8125 -1.890625 0.515625 -1.796875 0.515625 -1.421875 C 0.515625 -0.59375 1.359375 0.109375 2.46875 0.109375 C 3.71875 0.109375 4.703125 -0.78125 4.703125 -1.890625 Z M 4.703125 -1.890625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-9\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.78125 -1.734375 C 4.78125 -2.4375 4.3125 -3.03125 3.65625 -3.328125 C 4.21875 -3.6875 4.5 -4.25 4.5 -4.796875 C 4.5 -5.53125 3.765625 -6.203125 2.625 -6.203125 C 1.421875 -6.203125 0.734375 -5.71875 0.734375 -5.03125 C 0.734375 -4.703125 0.984375 -4.5625 1.171875 -4.5625 C 1.390625 -4.5625 1.609375 -4.734375 1.609375 -5.015625 C 1.609375 -5.15625 1.5625 -5.25 1.53125 -5.28125 C 1.828125 -5.59375 2.546875 -5.59375 2.625 -5.59375 C 3.3125 -5.59375 3.8125 -5.234375 3.8125 -4.78125 C 3.8125 -4.484375 3.65625 -4.140625 3.390625 -3.921875 C 3.078125 -3.65625 2.828125 -3.640625 2.46875 -3.625 C 1.890625 -3.578125 1.75 -3.578125 1.75 -3.296875 C 1.75 -2.984375 1.984375 -2.984375 2.140625 -2.984375 L 2.609375 -2.984375 C 3.59375 -2.984375 4.09375 -2.3125 4.09375 -1.734375 C 4.09375 -1.125 3.53125 -0.5 2.625 -0.5 C 2.234375 -0.5 1.46875 -0.609375 1.203125 -1.078125 C 1.25 -1.125 1.328125 -1.1875 1.328125 -1.390625 C 1.328125 -1.625 1.140625 -1.828125 0.890625 -1.828125 C 0.65625 -1.828125 0.4375 -1.671875 0.4375 -1.359375 C 0.4375 -0.46875 1.40625 0.109375 2.625 0.109375 C 3.9375 0.109375 4.78125 -0.8125 4.78125 -1.734375 Z M 4.78125 -1.734375 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656246-0-10\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.40625 -0.296875 C 4.40625 -0.609375 4.171875 -0.609375 4 -0.609375 L 3.09375 -0.609375 L 3.09375 -5.796875 C 3.09375 -5.953125 3.09375 -6.203125 2.796875 -6.203125 C 2.609375 -6.203125 2.546875 -6.078125 2.5 -5.96875 C 2.125 -5.109375 1.609375 -5 1.421875 -4.984375 C 1.25 -4.96875 1.046875 -4.953125 1.046875 -4.671875 C 1.046875 -4.421875 1.21875 -4.375 1.375 -4.375 C 1.5625 -4.375 1.96875 -4.4375 2.40625 -4.8125 L 2.40625 -0.609375 L 1.5 -0.609375 C 1.34375 -0.609375 1.109375 -0.609375 1.109375 -0.296875 C 1.109375 0 1.359375 0 1.5 0 L 4 0 C 4.15625 0 4.40625 0 4.40625 -0.296875 Z M 4.40625 -0.296875 \"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -3.081063 -6.166813 L -10.909188 -21.823063 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196287 1.591742 C -1.09495 0.996046 0.00039203 0.098159 0.299118 -0.0014074 C -0.00134959 -0.0992429 -1.094897 -0.995442 -1.196202 -1.596384 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,50.88113,27.78778)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 3.083 -6.166813 L 11.360344 -22.717594 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195082 1.594628 C -1.09727 0.995432 0.00151764 0.10098 0.300238 -0.000349055 C 0.00151225 -0.0999154 -1.097324 -0.996055 -1.195167 -1.593499 \" transform=\"matrix(0.44724,0.89442,0.89442,-0.44724,73.15119,28.68443)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -20.288094 -34.463688 L -36.530281 -50.701969 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19593 1.59239 C -1.096467 0.995798 0.000101202 0.0981938 0.298405 -0.00122439 C 0.000111045 -0.100672 -1.096369 -0.998385 -1.195772 -1.594987 \" transform=\"matrix(-0.70717,0.7071,0.7071,0.70717,25.26267,56.66877)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -14.174813 -34.463688 L -14.174813 -50.069156 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196556 1.592327 C -1.094994 0.994671 -0.00124375 0.10014 0.299538 -0.0014225 C -0.00124375 -0.0990788 -1.094994 -0.997516 -1.196556 -1.595173 \" transform=\"matrix(0,1,1,0,47.61861,56.0364)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -18.795906 -62.858219 L -30.155281 -77.998844 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19593 1.591343 C -1.096703 0.99682 0.000173386 0.100755 0.300953 -0.000801425 C -0.000604642 -0.100024 -1.096671 -0.995343 -1.195879 -1.592213 \" transform=\"matrix(-0.60002,0.8,0.8,0.60002,31.63825,83.96675)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -12.631844 -62.858219 L -8.741219 -78.428531 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193729 1.594006 C -1.094466 0.994733 -0.000812413 0.0986805 0.299773 -0.000561397 C 0.00202806 -0.0988726 -1.094418 -0.997813 -1.195543 -1.595201 \" transform=\"matrix(0.24307,0.97238,0.97238,-0.24307,53.05268,84.39587)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -10.170906 -91.205875 L -18.444344 -107.760563 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195846 1.591822 C -1.094509 0.996125 0.00083309 0.0982383 0.299559 -0.00132801 C -0.000908532 -0.0991635 -1.094456 -0.995362 -1.195761 -1.596305 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,43.3461,113.72485)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -4.002938 -91.205875 L 3.797844 -106.811344 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19513 1.593003 C -1.097685 0.996188 -0.00148642 0.100106 0.301272 0.000929191 C -0.00148464 -0.0999927 -1.097668 -0.996095 -1.196841 -1.594651 \" transform=\"matrix(0.44899,0.898,0.898,-0.44899,65.59046,112.77675)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -26.667 -118.604313 L -43.381844 -134.741031 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.197578 1.593439 C -1.096126 0.997339 -0.000247909 0.101946 0.296371 -0.000270765 C 0.000548994 -0.0991746 -1.096761 -0.99473 -1.193626 -1.594333 \" transform=\"matrix(-0.72139,0.69635,0.69635,0.72139,18.4093,140.70788)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -21.26075 -118.604313 L -21.26075 -134.604313 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19372 1.5931 C -1.096064 0.995444 0.0015925 0.100912 0.298468 -0.00065 C 0.0015925 -0.0983063 -1.096064 -0.996744 -1.19372 -1.5944 \" transform=\"matrix(0,1,1,0,40.5319,140.56872)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 7.086906 -119.553531 L 7.086906 -135.608219 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196044 1.593956 C -1.094481 0.9963 -0.00073125 0.0978625 0.30005 0.00020625 C -0.00073125 -0.101356 -1.094481 -0.995888 -1.196044 -1.593544 \" transform=\"matrix(0,1,1,0,68.8787,141.57495)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 13.223625 -119.522281 L 29.891594 -136.19025 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196473 1.595093 C -1.094273 0.995703 -0.000456972 0.100763 0.297857 0.00132495 C -0.000456972 -0.0981129 -1.097035 -0.995816 -1.196473 -1.592443 \" transform=\"matrix(0.7071,0.7071,0.7071,-0.7071,91.68298,142.15751)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 32.731437 -146.951969 L 24.418937 -162.998844 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19568 1.595722 C -1.096586 0.99807 -0.000577766 0.101193 0.296905 0.000953639 C -0.00117507 -0.100827 -1.094227 -0.995885 -1.197153 -1.592939 \" transform=\"matrix(-0.46143,0.89082,0.89082,0.46143,86.21037,168.96554)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 38.137687 -146.951969 L 46.450187 -162.998844 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196174 1.594829 C -1.095039 0.994317 -0.00198712 0.099259 0.297884 0.000936021 C 0.000401046 -0.0993032 -1.095607 -0.996181 -1.194701 -1.593833 \" transform=\"matrix(0.46143,0.89082,0.89082,-0.46143,108.24062,168.96554)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 14.172844 -33.56525 L 14.172844 -49.561344 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196259 1.593204 C -1.094696 0.995547 -0.00094625 0.101016 0.299835 -0.00054625 C -0.00094625 -0.0982025 -1.094696 -0.99664 -1.196259 -1.594296 \" transform=\"matrix(0,1,1,0,75.96539,55.52829)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 19.391594 -33.56525 L 36.079094 -50.25275 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193092 1.593886 C -1.096885 0.997481 -0.000245705 0.0987699 0.296586 -0.000167 C -0.000241528 -0.0991165 -1.096843 -0.997874 -1.195774 -1.591535 \" transform=\"matrix(0.71062,0.71065,0.71065,-0.71062,97.87139,56.2188)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 39.461906 -62.807438 L 31.887687 -77.955875 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196115 1.593653 C -1.094777 0.997956 0.000564714 0.100069 0.29929 0.000502758 C 0.000570105 -0.100827 -1.098218 -0.995278 -1.196029 -1.594474 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,93.68028,83.92349)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 45.797844 -62.807438 L 53.918937 -77.963688 \" transform=\"matrix(1,0,0,-1,61.792,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195093 1.594866 C -1.095246 0.997574 0.00194921 0.100023 0.297189 -0.000750864 C 0.00153865 -0.0986993 -1.094324 -0.996762 -1.1932 -1.593643 \" transform=\"matrix(0.47359,0.88358,0.88358,-0.47359,115.71148,83.92846)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-1\" x=\"59.177\" y=\"8.611\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-2\" x=\"45.003\" y=\"36.907\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-3\" x=\"16.657\" y=\"64.805\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-1\" x=\"45.003\" y=\"65.304\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-4\" x=\"18.513\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"23.743386\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"28.973772\" y=\"94.049\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-1\" x=\"52.09\" y=\"93.65\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-7\" x=\"37.917\" y=\"121.05\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-8\" x=\"3.321\" y=\"150.741\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"8.551386\" y=\"150.741\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"13.781772\" y=\"150.741\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-8\" x=\"32.686\" y=\"150.741\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"37.916386\" y=\"150.741\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"43.146772\" y=\"150.741\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-1\" x=\"66.263\" y=\"121.996\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-3\" x=\"66.263\" y=\"149.845\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-7\" x=\"94.61\" y=\"149.396\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-9\" x=\"74.696\" y=\"179.088\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"79.926386\" y=\"179.088\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"85.156772\" y=\"179.088\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-9\" x=\"104.062\" y=\"179.088\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"109.292386\" y=\"179.088\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"114.522772\" y=\"179.088\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-7\" x=\"73.35\" y=\"36.011\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-9\" x=\"68.119\" y=\"65.702\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"73.349386\" y=\"65.702\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"78.579772\" y=\"65.702\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-2\" x=\"101.696\" y=\"65.254\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-10\" x=\"82.293\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"87.523386\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"92.753772\" y=\"94.049\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-4\" x=\"111.658\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-5\" x=\"116.888386\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656246-0-6\" x=\"122.118772\" y=\"94.049\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n",
       "\n"
      ],
      "text/plain": [
       "TreeView.LabelledTree({19, 18} directed simple Int64 graph, Any[:+, :*, :x, :+, 2.0, :+, :-, 5.0, 5.0, :+, :x, :-, 3.0, 3.0, :-, 3.0, :*, 1.0, 2.0])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display(results_mc.tree, grammar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Genetic Programming\n",
    "\n",
    "Genetic Programming (GP) is an evolutionary algorithm for trees.\n",
    "\n",
    "See: Koza, \"Genetic Programming: On the Programming of Computers by Means of Natural Selection\", MIT Press, 1992."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mG\u001b[22m\u001b[1me\u001b[22m\u001b[1mn\u001b[22m\u001b[1me\u001b[22m\u001b[1mt\u001b[22m\u001b[1mi\u001b[22m\u001b[1mc\u001b[22m\u001b[1mP\u001b[22m\u001b[1mr\u001b[22m\u001b[1mo\u001b[22m\u001b[1mg\u001b[22m\u001b[1mr\u001b[22m\u001b[1ma\u001b[22m\u001b[1mm\u001b[22m \u001b[1mG\u001b[22m\u001b[1me\u001b[22m\u001b[1mn\u001b[22m\u001b[1me\u001b[22m\u001b[1mt\u001b[22m\u001b[1mi\u001b[22m\u001b[1mc\u001b[22m\u001b[1mP\u001b[22m\u001b[1mr\u001b[22m\u001b[1mo\u001b[22m\u001b[1mg\u001b[22m\u001b[1mr\u001b[22m\u001b[1ma\u001b[22m\u001b[1mm\u001b[22ms\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "GeneticProgram\n",
       "```\n",
       "\n",
       "Genetic Programming.\n",
       "\n",
       "# Arguments\n",
       "\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iterations::Int`: number of iterations\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n",
       "  * `p_reproduction::Float64`: probability of reproduction operator\n",
       "  * `p_crossover::Float64`: probability of crossover operator\n",
       "  * `p_mutation::Float64`: probability of mutation operator\n",
       "  * `init_method::InitializationMethod`: initialization method\n",
       "  * `select_method::SelectionMethod`: selection method\n"
      ],
      "text/plain": [
       "```\n",
       "GeneticProgram\n",
       "```\n",
       "\n",
       "Genetic Programming.\n",
       "\n",
       "# Arguments\n",
       "\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iterations::Int`: number of iterations\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n",
       "  * `p_reproduction::Float64`: probability of reproduction operator\n",
       "  * `p_crossover::Float64`: probability of crossover operator\n",
       "  * `p_mutation::Float64`: probability of mutation operator\n",
       "  * `init_method::InitializationMethod`: initialization method\n",
       "  * `select_method::SelectionMethod`: selection method\n"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?GeneticProgram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(:(x * x + ((x + 1.0) + x)), 0.0)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srand(1)\n",
    "p = GeneticProgram(1000,20,6,0.3,0.3,0.4)\n",
    "results_gp = optimize(p, grammar, :Real, loss)\n",
    "(results_gp.expr, results_gp.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"96.911pt\" height=\"97.479pt\" viewBox=\"0 0 96.911 97.479\" version=\"1.1\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-0\">\n",
       "<path style=\"stroke:none;\" d=\"\"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-1\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.84375 -3.046875 C 4.84375 -3.40625 4.5 -3.40625 4.375 -3.40625 L 2.953125 -3.40625 L 2.953125 -4.828125 C 2.953125 -4.9375 2.953125 -5.296875 2.609375 -5.296875 C 2.265625 -5.296875 2.265625 -4.9375 2.265625 -4.828125 L 2.265625 -3.40625 L 0.84375 -3.40625 C 0.734375 -3.40625 0.375 -3.40625 0.375 -3.046875 C 0.375 -2.703125 0.734375 -2.703125 0.84375 -2.703125 L 2.265625 -2.703125 L 2.265625 -1.28125 C 2.265625 -1.15625 2.265625 -0.8125 2.609375 -0.8125 C 2.953125 -0.8125 2.953125 -1.15625 2.953125 -1.28125 L 2.953125 -2.703125 L 4.375 -2.703125 C 4.5 -2.703125 4.84375 -2.703125 4.84375 -3.046875 Z M 4.84375 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-2\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.546875 -2.140625 C 4.546875 -2.296875 4.421875 -2.390625 4.40625 -2.40625 C 4.15625 -2.578125 3.484375 -2.90625 3.21875 -3.046875 L 4.296875 -3.625 C 4.421875 -3.703125 4.546875 -3.765625 4.546875 -3.953125 C 4.546875 -4 4.546875 -4.28125 4.140625 -4.28125 L 2.90625 -3.5625 C 2.9375 -3.828125 2.9375 -4.484375 2.9375 -4.78125 C 2.9375 -4.859375 2.9375 -5.1875 2.609375 -5.1875 C 2.28125 -5.1875 2.28125 -4.859375 2.28125 -4.78125 C 2.28125 -4.484375 2.296875 -3.828125 2.3125 -3.5625 L 1.234375 -4.203125 C 1.09375 -4.28125 1.078125 -4.28125 1 -4.28125 C 0.8125 -4.28125 0.671875 -4.109375 0.671875 -3.953125 C 0.671875 -3.765625 0.78125 -3.703125 0.921875 -3.640625 L 2 -3.046875 L 0.921875 -2.46875 C 0.8125 -2.390625 0.671875 -2.328125 0.671875 -2.140625 C 0.671875 -2.09375 0.671875 -1.796875 1.078125 -1.796875 L 2.3125 -2.515625 C 2.296875 -2.265625 2.28125 -1.609375 2.28125 -1.3125 C 2.28125 -1.21875 2.28125 -0.890625 2.609375 -0.890625 C 2.9375 -0.890625 2.9375 -1.21875 2.9375 -1.3125 C 2.9375 -1.609375 2.9375 -2.265625 2.90625 -2.515625 C 3.515625 -2.1875 3.15625 -2.375 3.84375 -1.96875 C 4.109375 -1.796875 4.140625 -1.796875 4.21875 -1.796875 C 4.421875 -1.796875 4.546875 -1.984375 4.546875 -2.140625 Z M 4.546875 -2.140625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-3\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.9375 -0.296875 C 4.9375 -0.609375 4.6875 -0.609375 4.53125 -0.609375 L 4.140625 -0.609375 L 2.859375 -2.21875 L 4 -3.6875 L 4.390625 -3.6875 C 4.53125 -3.6875 4.8125 -3.6875 4.8125 -3.984375 C 4.8125 -4.296875 4.546875 -4.296875 4.390625 -4.296875 L 3.234375 -4.296875 C 3.078125 -4.296875 2.828125 -4.296875 2.828125 -4 C 2.828125 -3.6875 3.046875 -3.6875 3.3125 -3.6875 L 2.578125 -2.6875 L 1.828125 -3.6875 C 2.078125 -3.6875 2.296875 -3.6875 2.296875 -4 C 2.296875 -4.296875 2.046875 -4.296875 1.90625 -4.296875 L 0.734375 -4.296875 C 0.59375 -4.296875 0.328125 -4.296875 0.328125 -3.984375 C 0.328125 -3.6875 0.59375 -3.6875 0.734375 -3.6875 L 1.140625 -3.6875 L 2.3125 -2.21875 L 1.078125 -0.609375 L 0.671875 -0.609375 C 0.53125 -0.609375 0.265625 -0.609375 0.265625 -0.296875 C 0.265625 0 0.53125 0 0.671875 0 L 1.84375 0 C 2 0 2.25 0 2.25 -0.296875 C 2.25 -0.609375 2.03125 -0.609375 1.71875 -0.609375 L 2.578125 -1.828125 L 3.46875 -0.609375 C 3.1875 -0.609375 2.96875 -0.609375 2.96875 -0.296875 C 2.96875 0 3.21875 0 3.375 0 L 4.53125 0 C 4.671875 0 4.9375 0 4.9375 -0.296875 Z M 4.9375 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-4\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.40625 -0.296875 C 4.40625 -0.609375 4.171875 -0.609375 4 -0.609375 L 3.09375 -0.609375 L 3.09375 -5.796875 C 3.09375 -5.953125 3.09375 -6.203125 2.796875 -6.203125 C 2.609375 -6.203125 2.546875 -6.078125 2.5 -5.96875 C 2.125 -5.109375 1.609375 -5 1.421875 -4.984375 C 1.25 -4.96875 1.046875 -4.953125 1.046875 -4.671875 C 1.046875 -4.421875 1.21875 -4.375 1.375 -4.375 C 1.5625 -4.375 1.96875 -4.4375 2.40625 -4.8125 L 2.40625 -0.609375 L 1.5 -0.609375 C 1.34375 -0.609375 1.109375 -0.609375 1.109375 -0.296875 C 1.109375 0 1.359375 0 1.5 0 L 4 0 C 4.15625 0 4.40625 0 4.40625 -0.296875 Z M 4.40625 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-5\">\n",
       "<path style=\"stroke:none;\" d=\"M 3.234375 -0.625 C 3.234375 -0.984375 2.9375 -1.25 2.625 -1.25 C 2.25 -1.25 2 -0.9375 2 -0.625 C 2 -0.265625 2.296875 0 2.609375 0 C 2.984375 0 3.234375 -0.3125 3.234375 -0.625 Z M 3.234375 -0.625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656247-0-6\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.71875 -3.046875 C 4.71875 -4.890625 3.703125 -6.203125 2.609375 -6.203125 C 1.5 -6.203125 0.5 -4.859375 0.5 -3.046875 C 0.5 -1.203125 1.515625 0.109375 2.609375 0.109375 C 3.734375 0.109375 4.71875 -1.21875 4.71875 -3.046875 Z M 4.03125 -3.15625 C 4.03125 -1.671875 3.390625 -0.5 2.609375 -0.5 C 1.828125 -0.5 1.1875 -1.671875 1.1875 -3.15625 C 1.1875 -4.609375 1.875 -5.59375 2.609375 -5.59375 C 3.34375 -5.59375 4.03125 -4.609375 4.03125 -3.15625 Z M 4.03125 -3.15625 \"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -3.083906 -6.166813 L -10.912031 -21.823063 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196763 1.592693 C -1.095425 0.996996 -0.0000831574 0.0991093 0.298643 -0.000457086 C -0.0000777668 -0.101786 -1.095372 -0.994491 -1.196677 -1.595434 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,37.54413,27.78778)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 3.084063 -6.166813 L 10.884844 -21.772281 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.197285 1.594392 C -1.09636 0.995838 -0.000161246 0.0997549 0.297377 -0.00116161 C -0.000159463 -0.100343 -1.096342 -0.996446 -1.195516 -1.595002 \" transform=\"matrix(0.44899,0.898,0.898,-0.44899,59.34018,27.73634)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -20.287031 -34.463688 L -36.529219 -50.701969 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196681 1.593142 C -1.097219 0.996549 -0.000650107 0.098945 0.297654 -0.000473152 C -0.000640264 -0.0999208 -1.09712 -0.997633 -1.196523 -1.594236 \" transform=\"matrix(-0.70717,0.7071,0.7071,0.70717,11.92567,56.66877)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -14.17375 -34.463688 L -14.17375 -50.569156 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.194696 1.59339 C -1.09704 0.995734 0.00061625 0.101202 0.297491 -0.00036 C 0.00061625 -0.0980163 -1.09704 -0.996454 -1.194696 -1.59411 \" transform=\"matrix(0,1,1,0,34.28161,56.53454)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 14.173906 -34.510563 L 14.173906 -50.069156 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196556 1.594266 C -1.094994 0.99661 -0.00124375 0.0981725 0.299538 0.00051625 C -0.00124375 -0.101046 -1.094994 -0.995577 -1.196556 -1.593234 \" transform=\"matrix(0,1,1,0,62.62839,56.0364)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 20.306719 -34.483219 L 36.529375 -50.701969 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196472 1.594355 C -1.097054 0.997742 -0.00053191 0.100033 0.29777 0.000589956 C -0.000537535 -0.0988366 -1.09711 -0.996483 -1.196562 -1.59309 \" transform=\"matrix(0.70714,0.7071,0.7071,-0.70714,84.98433,56.66877)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 11.091875 -62.858219 L 3.037188 -78.963688 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.194812 1.592805 C -1.095638 0.994249 0.000544863 0.0981469 0.298081 -0.00103495 C -0.00119687 -0.0984715 -1.095656 -0.998034 -1.1931 -1.594848 \" transform=\"matrix(-0.44899,0.898,0.898,0.44899,51.49414,84.9281)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 17.255938 -62.858219 L 24.802813 -77.955875 \" transform=\"matrix(1,0,0,-1,48.455,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196094 1.592283 C -1.093435 0.99721 0.00101058 0.097642 0.300287 0.000203229 C -0.000729399 -0.0972355 -1.095175 -0.998544 -1.196094 -1.591876 \" transform=\"matrix(0.449,0.898,0.898,-0.449,73.25952,83.92184)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-1\" x=\"45.84\" y=\"8.611\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-2\" x=\"31.667\" y=\"36.908\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-3\" x=\"3.321\" y=\"64.806\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-3\" x=\"31.667\" y=\"64.806\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-1\" x=\"60.013\" y=\"36.957\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-1\" x=\"60.013\" y=\"65.304\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-3\" x=\"45.84\" y=\"93.152\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-4\" x=\"68.956\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-5\" x=\"74.186386\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-6\" x=\"79.416772\" y=\"94.049\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656247-0-3\" x=\"88.36\" y=\"64.806\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n",
       "\n"
      ],
      "text/plain": [
       "TreeView.LabelledTree({9, 8} directed simple Int64 graph, Any[:+, :*, :x, :x, :+, :+, :x, 1.0, :x])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display(results_gp.tree, grammar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grammatical Evolution\n",
    "\n",
    "Grammatical Evolution (GE) is an evolutionary algorithm based on sequentializing the decisions in the derivation tree (e.g., using depth-first traversal order).  Optimization is performed over integer arrays using genetic algorithms.\n",
    "\n",
    "See: C. Ryan, J.J. Collins, M. O'Neil, \"Grammatical Evolution: Evolving Programs for an Arbitrary Language\", in European Conference on Genetic Programming, Springer, 1998, pp. 83-96."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mG\u001b[22m\u001b[1mr\u001b[22m\u001b[1ma\u001b[22m\u001b[1mm\u001b[22m\u001b[1mm\u001b[22m\u001b[1ma\u001b[22m\u001b[1mt\u001b[22m\u001b[1mi\u001b[22m\u001b[1mc\u001b[22m\u001b[1ma\u001b[22m\u001b[1ml\u001b[22m\u001b[1mE\u001b[22m\u001b[1mv\u001b[22m\u001b[1mo\u001b[22m\u001b[1ml\u001b[22m\u001b[1mu\u001b[22m\u001b[1mt\u001b[22m\u001b[1mi\u001b[22m\u001b[1mo\u001b[22m\u001b[1mn\u001b[22m \u001b[1mG\u001b[22m\u001b[1mr\u001b[22m\u001b[1ma\u001b[22m\u001b[1mm\u001b[22m\u001b[1mm\u001b[22m\u001b[1ma\u001b[22m\u001b[1mt\u001b[22m\u001b[1mi\u001b[22m\u001b[1mc\u001b[22m\u001b[1ma\u001b[22m\u001b[1ml\u001b[22m\u001b[1mE\u001b[22m\u001b[1mv\u001b[22m\u001b[1mo\u001b[22m\u001b[1ml\u001b[22m\u001b[1mu\u001b[22m\u001b[1mt\u001b[22m\u001b[1mi\u001b[22m\u001b[1mo\u001b[22m\u001b[1mn\u001b[22ms\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "GrammaticalEvolution\n",
       "```\n",
       "\n",
       "Grammatical Evolution.\n",
       "\n",
       "# Arguments\n",
       "\n",
       "  * `grammar::Grammar`: grammar\n",
       "  * `typ::Symbol`: start symbol\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iterations::Int`: number of iterations\n",
       "  * `init_gene_length::Int`: initial length of genotype integer array\n",
       "  * `max_gene_length::Int`: maximum length of genotype integer array\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n",
       "  * `p_reproduction::Float64`: probability of reproduction operator\n",
       "  * `p_crossover::Float64`: probability of crossover operator\n",
       "  * `p_mutation::Float64`: probability of mutation operator\n",
       "  * `select_method::SelectionMethod`: selection method (default: tournament selection)\n",
       "  * `mutate_method::InitializationMethod`: mutation method (default: multi-mutate)\n"
      ],
      "text/plain": [
       "```\n",
       "GrammaticalEvolution\n",
       "```\n",
       "\n",
       "Grammatical Evolution.\n",
       "\n",
       "# Arguments\n",
       "\n",
       "  * `grammar::Grammar`: grammar\n",
       "  * `typ::Symbol`: start symbol\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iterations::Int`: number of iterations\n",
       "  * `init_gene_length::Int`: initial length of genotype integer array\n",
       "  * `max_gene_length::Int`: maximum length of genotype integer array\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n",
       "  * `p_reproduction::Float64`: probability of reproduction operator\n",
       "  * `p_crossover::Float64`: probability of crossover operator\n",
       "  * `p_mutation::Float64`: probability of mutation operator\n",
       "  * `select_method::SelectionMethod`: selection method (default: tournament selection)\n",
       "  * `mutate_method::InitializationMethod`: mutation method (default: multi-mutate)\n"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?GrammaticalEvolution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(:(((2.0 + x) - 1.0) * (1.0 + x)), 0.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srand(0)\n",
    "p = GrammaticalEvolution(grammar,:Real,1000,20,10,10,6,0.2,0.4,0.4; select_method=GrammaticalEvolutions.TruncationSelection(300))\n",
    "results_ge = optimize(p, grammar, :Real, loss)\n",
    "(results_ge.expr, results_ge.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"117.334pt\" height=\"97.429pt\" viewBox=\"0 0 117.334 97.429\" version=\"1.1\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-0\">\n",
       "<path style=\"stroke:none;\" d=\"\"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-1\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.546875 -2.140625 C 4.546875 -2.296875 4.421875 -2.390625 4.40625 -2.40625 C 4.15625 -2.578125 3.484375 -2.90625 3.21875 -3.046875 L 4.296875 -3.625 C 4.421875 -3.703125 4.546875 -3.765625 4.546875 -3.953125 C 4.546875 -4 4.546875 -4.28125 4.140625 -4.28125 L 2.90625 -3.5625 C 2.9375 -3.828125 2.9375 -4.484375 2.9375 -4.78125 C 2.9375 -4.859375 2.9375 -5.1875 2.609375 -5.1875 C 2.28125 -5.1875 2.28125 -4.859375 2.28125 -4.78125 C 2.28125 -4.484375 2.296875 -3.828125 2.3125 -3.5625 L 1.234375 -4.203125 C 1.09375 -4.28125 1.078125 -4.28125 1 -4.28125 C 0.8125 -4.28125 0.671875 -4.109375 0.671875 -3.953125 C 0.671875 -3.765625 0.78125 -3.703125 0.921875 -3.640625 L 2 -3.046875 L 0.921875 -2.46875 C 0.8125 -2.390625 0.671875 -2.328125 0.671875 -2.140625 C 0.671875 -2.09375 0.671875 -1.796875 1.078125 -1.796875 L 2.3125 -2.515625 C 2.296875 -2.265625 2.28125 -1.609375 2.28125 -1.3125 C 2.28125 -1.21875 2.28125 -0.890625 2.609375 -0.890625 C 2.9375 -0.890625 2.9375 -1.21875 2.9375 -1.3125 C 2.9375 -1.609375 2.9375 -2.265625 2.90625 -2.515625 C 3.515625 -2.1875 3.15625 -2.375 3.84375 -1.96875 C 4.109375 -1.796875 4.140625 -1.796875 4.21875 -1.796875 C 4.421875 -1.796875 4.546875 -1.984375 4.546875 -2.140625 Z M 4.546875 -2.140625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-2\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.671875 -3.046875 C 4.671875 -3.40625 4.3125 -3.40625 4.203125 -3.40625 L 1.03125 -3.40625 C 0.90625 -3.40625 0.5625 -3.40625 0.5625 -3.046875 C 0.5625 -2.703125 0.90625 -2.703125 1.03125 -2.703125 L 4.203125 -2.703125 C 4.3125 -2.703125 4.671875 -2.703125 4.671875 -3.046875 Z M 4.671875 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-3\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.84375 -3.046875 C 4.84375 -3.40625 4.5 -3.40625 4.375 -3.40625 L 2.953125 -3.40625 L 2.953125 -4.828125 C 2.953125 -4.9375 2.953125 -5.296875 2.609375 -5.296875 C 2.265625 -5.296875 2.265625 -4.9375 2.265625 -4.828125 L 2.265625 -3.40625 L 0.84375 -3.40625 C 0.734375 -3.40625 0.375 -3.40625 0.375 -3.046875 C 0.375 -2.703125 0.734375 -2.703125 0.84375 -2.703125 L 2.265625 -2.703125 L 2.265625 -1.28125 C 2.265625 -1.15625 2.265625 -0.8125 2.609375 -0.8125 C 2.953125 -0.8125 2.953125 -1.15625 2.953125 -1.28125 L 2.953125 -2.703125 L 4.375 -2.703125 C 4.5 -2.703125 4.84375 -2.703125 4.84375 -3.046875 Z M 4.84375 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-4\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.703125 -0.40625 L 4.703125 -0.671875 C 4.703125 -0.859375 4.703125 -1.078125 4.359375 -1.078125 C 4.015625 -1.078125 4.015625 -0.890625 4.015625 -0.609375 L 1.640625 -0.609375 C 2.234375 -1.109375 3.1875 -1.859375 3.625 -2.265625 C 4.25 -2.828125 4.703125 -3.453125 4.703125 -4.25 C 4.703125 -5.453125 3.703125 -6.203125 2.484375 -6.203125 C 1.3125 -6.203125 0.515625 -5.390625 0.515625 -4.53125 C 0.515625 -4.171875 0.796875 -4.0625 0.96875 -4.0625 C 1.171875 -4.0625 1.40625 -4.234375 1.40625 -4.5 C 1.40625 -4.625 1.359375 -4.75 1.265625 -4.828125 C 1.421875 -5.28125 1.890625 -5.59375 2.4375 -5.59375 C 3.25 -5.59375 4.015625 -5.140625 4.015625 -4.25 C 4.015625 -3.5625 3.53125 -2.984375 2.875 -2.4375 L 0.671875 -0.578125 C 0.578125 -0.5 0.515625 -0.453125 0.515625 -0.3125 C 0.515625 0 0.765625 0 0.921875 0 L 4.3125 0 C 4.640625 0 4.703125 -0.09375 4.703125 -0.40625 Z M 4.703125 -0.40625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-5\">\n",
       "<path style=\"stroke:none;\" d=\"M 3.234375 -0.625 C 3.234375 -0.984375 2.9375 -1.25 2.625 -1.25 C 2.25 -1.25 2 -0.9375 2 -0.625 C 2 -0.265625 2.296875 0 2.609375 0 C 2.984375 0 3.234375 -0.3125 3.234375 -0.625 Z M 3.234375 -0.625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-6\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.71875 -3.046875 C 4.71875 -4.890625 3.703125 -6.203125 2.609375 -6.203125 C 1.5 -6.203125 0.5 -4.859375 0.5 -3.046875 C 0.5 -1.203125 1.515625 0.109375 2.609375 0.109375 C 3.734375 0.109375 4.71875 -1.21875 4.71875 -3.046875 Z M 4.03125 -3.15625 C 4.03125 -1.671875 3.390625 -0.5 2.609375 -0.5 C 1.828125 -0.5 1.1875 -1.671875 1.1875 -3.15625 C 1.1875 -4.609375 1.875 -5.59375 2.609375 -5.59375 C 3.34375 -5.59375 4.03125 -4.609375 4.03125 -3.15625 Z M 4.03125 -3.15625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-7\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.9375 -0.296875 C 4.9375 -0.609375 4.6875 -0.609375 4.53125 -0.609375 L 4.140625 -0.609375 L 2.859375 -2.21875 L 4 -3.6875 L 4.390625 -3.6875 C 4.53125 -3.6875 4.8125 -3.6875 4.8125 -3.984375 C 4.8125 -4.296875 4.546875 -4.296875 4.390625 -4.296875 L 3.234375 -4.296875 C 3.078125 -4.296875 2.828125 -4.296875 2.828125 -4 C 2.828125 -3.6875 3.046875 -3.6875 3.3125 -3.6875 L 2.578125 -2.6875 L 1.828125 -3.6875 C 2.078125 -3.6875 2.296875 -3.6875 2.296875 -4 C 2.296875 -4.296875 2.046875 -4.296875 1.90625 -4.296875 L 0.734375 -4.296875 C 0.59375 -4.296875 0.328125 -4.296875 0.328125 -3.984375 C 0.328125 -3.6875 0.59375 -3.6875 0.734375 -3.6875 L 1.140625 -3.6875 L 2.3125 -2.21875 L 1.078125 -0.609375 L 0.671875 -0.609375 C 0.53125 -0.609375 0.265625 -0.609375 0.265625 -0.296875 C 0.265625 0 0.53125 0 0.671875 0 L 1.84375 0 C 2 0 2.25 0 2.25 -0.296875 C 2.25 -0.609375 2.03125 -0.609375 1.71875 -0.609375 L 2.578125 -1.828125 L 3.46875 -0.609375 C 3.1875 -0.609375 2.96875 -0.609375 2.96875 -0.296875 C 2.96875 0 3.21875 0 3.375 0 L 4.53125 0 C 4.671875 0 4.9375 0 4.9375 -0.296875 Z M 4.9375 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656248-0-8\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.40625 -0.296875 C 4.40625 -0.609375 4.171875 -0.609375 4 -0.609375 L 3.09375 -0.609375 L 3.09375 -5.796875 C 3.09375 -5.953125 3.09375 -6.203125 2.796875 -6.203125 C 2.609375 -6.203125 2.546875 -6.078125 2.5 -5.96875 C 2.125 -5.109375 1.609375 -5 1.421875 -4.984375 C 1.25 -4.96875 1.046875 -4.953125 1.046875 -4.671875 C 1.046875 -4.421875 1.21875 -4.375 1.375 -4.375 C 1.5625 -4.375 1.96875 -4.4375 2.40625 -4.8125 L 2.40625 -0.609375 L 1.5 -0.609375 C 1.34375 -0.609375 1.109375 -0.609375 1.109375 -0.296875 C 1.109375 0 1.359375 0 1.5 0 L 4 0 C 4.15625 0 4.40625 0 4.40625 -0.296875 Z M 4.40625 -0.296875 \"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -3.165875 -6.11525 L -11.767437 -22.720719 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196398 1.59164 C -1.093861 0.995768 0.00213831 0.0988489 0.299622 -0.00139959 C -0.00025516 -0.099716 -1.096803 -0.996548 -1.194498 -1.595275 \" transform=\"matrix(-0.4614,0.89082,0.89082,0.4614,56.60043,28.63608)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 3.166156 -6.11525 L 11.279438 -21.775406 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.194115 1.59332 C -1.096454 0.994605 0.0000155679 0.097739 0.29988 -0.000590648 C -0.00105885 -0.0990297 -1.097076 -0.995866 -1.196184 -1.593507 \" transform=\"matrix(0.46146,0.89082,0.89082,-0.46146,79.6473,27.68962)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -19.90025 -33.564469 L -36.572125 -50.236344 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193797 1.594142 C -1.097121 0.994752 0.00221945 0.0998117 0.300533 0.000373886 C -0.00054271 -0.101826 -1.094359 -0.996767 -1.193797 -1.593394 \" transform=\"matrix(-0.7071,0.7071,0.7071,0.7071,31.79818,56.15051)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -14.6815 -33.564469 L -14.6815 -49.560563 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19704 1.59534 C -1.095478 0.997684 -0.0017275 0.0992463 0.299054 0.00159 C -0.0017275 -0.0999725 -1.095478 -0.994504 -1.19704 -1.59216 \" transform=\"matrix(0,1,1,0,53.68591,55.47829)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -46.111187 -62.857438 L -53.661969 -77.955094 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193862 1.594373 C -1.094684 0.995821 0.00150223 0.0997322 0.299038 0.000553452 C 0.00150223 -0.100365 -1.094684 -0.996454 -1.195602 -1.595006 \" transform=\"matrix(-0.449,0.898,0.898,0.449,14.70799,83.87184)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -39.947125 -62.857438 L -31.896344 -78.962906 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.194971 1.592848 C -1.097526 0.996034 -0.00132718 0.0999509 0.297951 0.00251436 C -0.00132539 -0.100147 -1.097508 -0.99625 -1.196682 -1.594806 \" transform=\"matrix(0.44899,0.898,0.898,-0.44899,36.47334,84.8781)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 14.681781 -34.509781 L 14.681781 -49.560563 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19704 1.592441 C -1.095478 0.994785 -0.0017275 0.100254 0.299054 -0.00130875 C -0.0017275 -0.098965 -1.095478 -0.997402 -1.19704 -1.595059 \" transform=\"matrix(0,1,1,0,83.05209,55.47829)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 20.8185 -34.482438 L 37.03725 -50.701188 \" transform=\"matrix(1,0,0,-1,68.369,5.916)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195553 1.596379 C -1.096135 0.994242 -0.00237479 0.0992954 0.301452 -0.000148177 C -0.00238042 -0.0995746 -1.096191 -0.994459 -1.195643 -1.596591 \" transform=\"matrix(0.70714,0.7071,0.7071,-0.70714,105.40803,56.61877)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-1\" x=\"65.753\" y=\"8.511\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-2\" x=\"51.071\" y=\"35.961\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-3\" x=\"22.724\" y=\"65.254\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-4\" x=\"3.321\" y=\"93.999\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-5\" x=\"8.551386\" y=\"93.999\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-6\" x=\"13.781772\" y=\"93.999\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-7\" x=\"36.897\" y=\"93.102\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-8\" x=\"45.84\" y=\"65.652\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-5\" x=\"51.070386\" y=\"65.652\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-6\" x=\"56.300772\" y=\"65.652\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-3\" x=\"80.436\" y=\"36.907\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-8\" x=\"75.206\" y=\"65.652\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-5\" x=\"80.436386\" y=\"65.652\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-6\" x=\"85.666772\" y=\"65.652\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656248-0-7\" x=\"108.783\" y=\"64.756\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n",
       "\n"
      ],
      "text/plain": [
       "TreeView.LabelledTree({9, 8} directed simple Int64 graph, Any[:*, :-, :+, 2.0, :x, 1.0, :+, 1.0, :x])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display(results_ge.tree, grammar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cross-Entropy Method\n",
    "\n",
    "The Cross-Entropy (CE) Method is a population-based optimization algorithm based on repeatedly estimating the probability distribution of good solutions.  This implementation uses a probabilistic grammar to represent the distributions.\n",
    "\n",
    "See: Rubinstein, \"Optimization of Computer Simulation Models with Rare Events\", European Journal of Operations Research, 99, 89-112, 1197"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mC\u001b[22m\u001b[1mr\u001b[22m\u001b[1mo\u001b[22m\u001b[1ms\u001b[22m\u001b[1ms\u001b[22m\u001b[1mE\u001b[22m\u001b[1mn\u001b[22m\u001b[1mt\u001b[22m\u001b[1mr\u001b[22m\u001b[1mo\u001b[22m\u001b[1mp\u001b[22m\u001b[1my\u001b[22m \u001b[1mC\u001b[22m\u001b[1mr\u001b[22m\u001b[1mo\u001b[22m\u001b[1ms\u001b[22m\u001b[1ms\u001b[22m\u001b[1mE\u001b[22m\u001b[1mn\u001b[22m\u001b[1mt\u001b[22m\u001b[1mr\u001b[22m\u001b[1mo\u001b[22m\u001b[1mp\u001b[22m\u001b[1my\u001b[22ms\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "CrossEntropy\n",
       "```\n",
       "\n",
       "Cross Entropy method.\n",
       "\n",
       "# Arguments\n",
       "\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iiterations::Int`: number of iterations\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n",
       "  * `top_k::Int`: top k elite samples used in selection\n",
       "  * `p_init::Float64`: initial value when fitting MLE\n",
       "  * `init_method::InitializationMethod`: Initialization method\n"
      ],
      "text/plain": [
       "```\n",
       "CrossEntropy\n",
       "```\n",
       "\n",
       "Cross Entropy method.\n",
       "\n",
       "# Arguments\n",
       "\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iiterations::Int`: number of iterations\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n",
       "  * `top_k::Int`: top k elite samples used in selection\n",
       "  * `p_init::Float64`: initial value when fitting MLE\n",
       "  * `init_method::InitializationMethod`: Initialization method\n"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?CrossEntropy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(:(x * (2.0 + x) + 1.0), 0.0)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srand(1)\n",
    "p = CrossEntropy(1000,20,6,500)\n",
    "results_ce = optimize(p, grammar, :Real, loss)\n",
    "(results_ce.expr, results_ce.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"59.621pt\" height=\"97.479pt\" viewBox=\"0 0 59.621 97.479\" version=\"1.1\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-0\">\n",
       "<path style=\"stroke:none;\" d=\"\"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-1\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.84375 -3.046875 C 4.84375 -3.40625 4.5 -3.40625 4.375 -3.40625 L 2.953125 -3.40625 L 2.953125 -4.828125 C 2.953125 -4.9375 2.953125 -5.296875 2.609375 -5.296875 C 2.265625 -5.296875 2.265625 -4.9375 2.265625 -4.828125 L 2.265625 -3.40625 L 0.84375 -3.40625 C 0.734375 -3.40625 0.375 -3.40625 0.375 -3.046875 C 0.375 -2.703125 0.734375 -2.703125 0.84375 -2.703125 L 2.265625 -2.703125 L 2.265625 -1.28125 C 2.265625 -1.15625 2.265625 -0.8125 2.609375 -0.8125 C 2.953125 -0.8125 2.953125 -1.15625 2.953125 -1.28125 L 2.953125 -2.703125 L 4.375 -2.703125 C 4.5 -2.703125 4.84375 -2.703125 4.84375 -3.046875 Z M 4.84375 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-2\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.546875 -2.140625 C 4.546875 -2.296875 4.421875 -2.390625 4.40625 -2.40625 C 4.15625 -2.578125 3.484375 -2.90625 3.21875 -3.046875 L 4.296875 -3.625 C 4.421875 -3.703125 4.546875 -3.765625 4.546875 -3.953125 C 4.546875 -4 4.546875 -4.28125 4.140625 -4.28125 L 2.90625 -3.5625 C 2.9375 -3.828125 2.9375 -4.484375 2.9375 -4.78125 C 2.9375 -4.859375 2.9375 -5.1875 2.609375 -5.1875 C 2.28125 -5.1875 2.28125 -4.859375 2.28125 -4.78125 C 2.28125 -4.484375 2.296875 -3.828125 2.3125 -3.5625 L 1.234375 -4.203125 C 1.09375 -4.28125 1.078125 -4.28125 1 -4.28125 C 0.8125 -4.28125 0.671875 -4.109375 0.671875 -3.953125 C 0.671875 -3.765625 0.78125 -3.703125 0.921875 -3.640625 L 2 -3.046875 L 0.921875 -2.46875 C 0.8125 -2.390625 0.671875 -2.328125 0.671875 -2.140625 C 0.671875 -2.09375 0.671875 -1.796875 1.078125 -1.796875 L 2.3125 -2.515625 C 2.296875 -2.265625 2.28125 -1.609375 2.28125 -1.3125 C 2.28125 -1.21875 2.28125 -0.890625 2.609375 -0.890625 C 2.9375 -0.890625 2.9375 -1.21875 2.9375 -1.3125 C 2.9375 -1.609375 2.9375 -2.265625 2.90625 -2.515625 C 3.515625 -2.1875 3.15625 -2.375 3.84375 -1.96875 C 4.109375 -1.796875 4.140625 -1.796875 4.21875 -1.796875 C 4.421875 -1.796875 4.546875 -1.984375 4.546875 -2.140625 Z M 4.546875 -2.140625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-3\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.9375 -0.296875 C 4.9375 -0.609375 4.6875 -0.609375 4.53125 -0.609375 L 4.140625 -0.609375 L 2.859375 -2.21875 L 4 -3.6875 L 4.390625 -3.6875 C 4.53125 -3.6875 4.8125 -3.6875 4.8125 -3.984375 C 4.8125 -4.296875 4.546875 -4.296875 4.390625 -4.296875 L 3.234375 -4.296875 C 3.078125 -4.296875 2.828125 -4.296875 2.828125 -4 C 2.828125 -3.6875 3.046875 -3.6875 3.3125 -3.6875 L 2.578125 -2.6875 L 1.828125 -3.6875 C 2.078125 -3.6875 2.296875 -3.6875 2.296875 -4 C 2.296875 -4.296875 2.046875 -4.296875 1.90625 -4.296875 L 0.734375 -4.296875 C 0.59375 -4.296875 0.328125 -4.296875 0.328125 -3.984375 C 0.328125 -3.6875 0.59375 -3.6875 0.734375 -3.6875 L 1.140625 -3.6875 L 2.3125 -2.21875 L 1.078125 -0.609375 L 0.671875 -0.609375 C 0.53125 -0.609375 0.265625 -0.609375 0.265625 -0.296875 C 0.265625 0 0.53125 0 0.671875 0 L 1.84375 0 C 2 0 2.25 0 2.25 -0.296875 C 2.25 -0.609375 2.03125 -0.609375 1.71875 -0.609375 L 2.578125 -1.828125 L 3.46875 -0.609375 C 3.1875 -0.609375 2.96875 -0.609375 2.96875 -0.296875 C 2.96875 0 3.21875 0 3.375 0 L 4.53125 0 C 4.671875 0 4.9375 0 4.9375 -0.296875 Z M 4.9375 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-4\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.703125 -0.40625 L 4.703125 -0.671875 C 4.703125 -0.859375 4.703125 -1.078125 4.359375 -1.078125 C 4.015625 -1.078125 4.015625 -0.890625 4.015625 -0.609375 L 1.640625 -0.609375 C 2.234375 -1.109375 3.1875 -1.859375 3.625 -2.265625 C 4.25 -2.828125 4.703125 -3.453125 4.703125 -4.25 C 4.703125 -5.453125 3.703125 -6.203125 2.484375 -6.203125 C 1.3125 -6.203125 0.515625 -5.390625 0.515625 -4.53125 C 0.515625 -4.171875 0.796875 -4.0625 0.96875 -4.0625 C 1.171875 -4.0625 1.40625 -4.234375 1.40625 -4.5 C 1.40625 -4.625 1.359375 -4.75 1.265625 -4.828125 C 1.421875 -5.28125 1.890625 -5.59375 2.4375 -5.59375 C 3.25 -5.59375 4.015625 -5.140625 4.015625 -4.25 C 4.015625 -3.5625 3.53125 -2.984375 2.875 -2.4375 L 0.671875 -0.578125 C 0.578125 -0.5 0.515625 -0.453125 0.515625 -0.3125 C 0.515625 0 0.765625 0 0.921875 0 L 4.3125 0 C 4.640625 0 4.703125 -0.09375 4.703125 -0.40625 Z M 4.703125 -0.40625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-5\">\n",
       "<path style=\"stroke:none;\" d=\"M 3.234375 -0.625 C 3.234375 -0.984375 2.9375 -1.25 2.625 -1.25 C 2.25 -1.25 2 -0.9375 2 -0.625 C 2 -0.265625 2.296875 0 2.609375 0 C 2.984375 0 3.234375 -0.3125 3.234375 -0.625 Z M 3.234375 -0.625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-6\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.71875 -3.046875 C 4.71875 -4.890625 3.703125 -6.203125 2.609375 -6.203125 C 1.5 -6.203125 0.5 -4.859375 0.5 -3.046875 C 0.5 -1.203125 1.515625 0.109375 2.609375 0.109375 C 3.734375 0.109375 4.71875 -1.21875 4.71875 -3.046875 Z M 4.03125 -3.15625 C 4.03125 -1.671875 3.390625 -0.5 2.609375 -0.5 C 1.828125 -0.5 1.1875 -1.671875 1.1875 -3.15625 C 1.1875 -4.609375 1.875 -5.59375 2.609375 -5.59375 C 3.34375 -5.59375 4.03125 -4.609375 4.03125 -3.15625 Z M 4.03125 -3.15625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656249-0-7\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.40625 -0.296875 C 4.40625 -0.609375 4.171875 -0.609375 4 -0.609375 L 3.09375 -0.609375 L 3.09375 -5.796875 C 3.09375 -5.953125 3.09375 -6.203125 2.796875 -6.203125 C 2.609375 -6.203125 2.546875 -6.078125 2.5 -5.96875 C 2.125 -5.109375 1.609375 -5 1.421875 -4.984375 C 1.25 -4.96875 1.046875 -4.953125 1.046875 -4.671875 C 1.046875 -4.421875 1.21875 -4.375 1.375 -4.375 C 1.5625 -4.375 1.96875 -4.4375 2.40625 -4.8125 L 2.40625 -0.609375 L 1.5 -0.609375 C 1.34375 -0.609375 1.109375 -0.609375 1.109375 -0.296875 C 1.109375 0 1.359375 0 1.5 0 L 4 0 C 4.15625 0 4.40625 0 4.40625 -0.296875 Z M 4.40625 -0.296875 \"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -3.082781 -6.166813 L -10.910906 -21.823063 \" transform=\"matrix(1,0,0,-1,34.282,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.197266 1.593699 C -1.095928 0.998002 -0.000586297 0.100115 0.298139 0.000549125 C -0.000580906 -0.10078 -1.094128 -0.996979 -1.19718 -1.594428 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,23.37113,27.78778)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 3.081281 -6.166813 L 10.632062 -21.260563 \" transform=\"matrix(1,0,0,-1,34.282,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19643 1.595424 C -1.095512 0.996872 0.000674276 0.100783 0.29821 -0.000135301 C 0.000674276 -0.099314 -1.095512 -0.995403 -1.19469 -1.593955 \" transform=\"matrix(0.449,0.898,0.898,-0.449,44.9131,27.22824)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -17.231219 -34.463688 L -25.309344 -50.616031 \" transform=\"matrix(1,0,0,-1,34.282,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195335 1.593694 C -1.097491 0.996251 0.00134454 0.100111 0.30007 0.000544705 C 0.00134993 -0.100785 -1.097438 -0.995236 -1.195249 -1.594432 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,8.97356,56.58293)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -11.114031 -34.463688 L -3.285906 -50.119938 \" transform=\"matrix(1,0,0,-1,34.282,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196649 1.595792 C -1.095344 0.994849 -0.00179682 0.0986504 0.298671 0.000814854 C -0.0000551971 -0.0987515 -1.095397 -0.996638 -1.196735 -1.592335 \" transform=\"matrix(0.44724,0.89442,0.89442,-0.44724,30.9946,56.08479)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -3.082781 -62.858219 L -10.629656 -77.955875 \" transform=\"matrix(1,0,0,-1,34.282,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195977 1.591641 C -1.095058 0.998309 -0.00235189 0.100481 0.300404 -0.000438196 C 0.00112806 -0.0978769 -1.095058 -0.993965 -1.195977 -1.592518 \" transform=\"matrix(-0.449,0.898,0.898,0.449,23.6509,83.92184)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 3.081281 -62.858219 L 11.135969 -78.963688 \" transform=\"matrix(1,0,0,-1,34.282,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193204 1.59464 C -1.09576 0.997826 -0.00130099 0.0982632 0.297977 0.000826712 C 0.000440745 -0.0983551 -1.095742 -0.994457 -1.194916 -1.593013 \" transform=\"matrix(0.44899,0.898,0.898,-0.44899,45.41625,84.9281)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-1\" x=\"31.667\" y=\"8.611\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-2\" x=\"17.494\" y=\"36.908\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-3\" x=\"3.321\" y=\"64.806\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-1\" x=\"31.667\" y=\"65.304\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-4\" x=\"12.263\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-5\" x=\"17.493386\" y=\"94.049\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-6\" x=\"22.723772\" y=\"94.049\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-3\" x=\"45.84\" y=\"93.152\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-7\" x=\"40.61\" y=\"37.356\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-5\" x=\"45.840386\" y=\"37.356\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656249-0-6\" x=\"51.070772\" y=\"37.356\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n",
       "\n"
      ],
      "text/plain": [
       "TreeView.LabelledTree({7, 6} directed simple Int64 graph, Any[:+, :*, :x, :+, 2.0, :x, 1.0])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display(results_ce.tree, grammar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PIPE\n",
    "\n",
    "Probabilistic Incremental Program Evolution (PIPE) is an expression tree optimization algorithm based on the probabilistic prototype tree (PPT) model.\n",
    "\n",
    "See: Salustowicz and Schmidhuber, \"Probabilistic Incremental Program Evolution\", Evolutionary Computation, vol. 5, no. 2, pp. 123-141, 1997."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "search: \u001b[1mP\u001b[22m\u001b[1mI\u001b[22m\u001b[1mP\u001b[22m\u001b[1mE\u001b[22m \u001b[1mP\u001b[22m\u001b[1mI\u001b[22m\u001b[1mP\u001b[22m\u001b[1mE\u001b[22ms \u001b[1mP\u001b[22m\u001b[1mi\u001b[22m\u001b[1mp\u001b[22m\u001b[1me\u001b[22m \u001b[1mp\u001b[22m\u001b[1mi\u001b[22m\u001b[1mp\u001b[22m\u001b[1me\u001b[22mline \u001b[1mP\u001b[22m\u001b[1mi\u001b[22m\u001b[1mp\u001b[22m\u001b[1me\u001b[22mBuffer Base64Encode\u001b[1mP\u001b[22m\u001b[1mi\u001b[22m\u001b[1mp\u001b[22m\u001b[1me\u001b[22m Base64Decode\u001b[1mP\u001b[22m\u001b[1mi\u001b[22m\u001b[1mp\u001b[22m\u001b[1me\u001b[22m\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```\n",
       "PIPE\n",
       "```\n",
       "\n",
       "Probabilistic Incremental Program Evolution. Example parameters from paper are indicated in parentheses)\n",
       "\n",
       "# Arguments:\n",
       "\n",
       "  * `ppt_params::PPT`: parameters for PPT  (e.g., [0.8, 0.2])\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iterations::Int`: number of iterations\n",
       "  * `p_elitist::Float64`: elitist update probability (e.g., 0.2)\n",
       "  * `c::Float64`: learning rate multiplier (e.g., 0.1)\n",
       "  * `α::Float64`: learning rate (e.g., 0.05)\n",
       "  * `ϵ::Float64`: fitness constant (e.g., 1)\n",
       "  * `p_mutation::Float64`: mutation probability (e.g., 0.2)\n",
       "  * `β::Float64`: mutation rate (e.g., 0.6)\n",
       "  * `p_threshold::Float64`: prune threshold (e.g., 0.999)\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n"
      ],
      "text/plain": [
       "```\n",
       "PIPE\n",
       "```\n",
       "\n",
       "Probabilistic Incremental Program Evolution. Example parameters from paper are indicated in parentheses)\n",
       "\n",
       "# Arguments:\n",
       "\n",
       "  * `ppt_params::PPT`: parameters for PPT  (e.g., [0.8, 0.2])\n",
       "  * `pop_size::Int`: population size\n",
       "  * `iterations::Int`: number of iterations\n",
       "  * `p_elitist::Float64`: elitist update probability (e.g., 0.2)\n",
       "  * `c::Float64`: learning rate multiplier (e.g., 0.1)\n",
       "  * `α::Float64`: learning rate (e.g., 0.05)\n",
       "  * `ϵ::Float64`: fitness constant (e.g., 1)\n",
       "  * `p_mutation::Float64`: mutation probability (e.g., 0.2)\n",
       "  * `β::Float64`: mutation rate (e.g., 0.6)\n",
       "  * `p_threshold::Float64`: prune threshold (e.g., 0.999)\n",
       "  * `max_depth::Int`: maximum depth of derivation tree\n"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "?PIPE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(:(x * (1.0x - ((5.0 - 1.0) - 5.0)) + (1.0 + x)), 0.0)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "srand(3)\n",
    "p = PIPE(PPT(0.8),1000,20,0.2,0.1,0.05,1,0.2,0.6,0.999,6)\n",
    "results_pipe = optimize(p, grammar, :Real, loss)\n",
    "(results_pipe.expr, results_pipe.loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"116.314pt\" height=\"154.172pt\" viewBox=\"0 0 116.314 154.172\" version=\"1.1\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-0\">\n",
       "<path style=\"stroke:none;\" d=\"\"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-1\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.84375 -3.046875 C 4.84375 -3.40625 4.5 -3.40625 4.375 -3.40625 L 2.953125 -3.40625 L 2.953125 -4.828125 C 2.953125 -4.9375 2.953125 -5.296875 2.609375 -5.296875 C 2.265625 -5.296875 2.265625 -4.9375 2.265625 -4.828125 L 2.265625 -3.40625 L 0.84375 -3.40625 C 0.734375 -3.40625 0.375 -3.40625 0.375 -3.046875 C 0.375 -2.703125 0.734375 -2.703125 0.84375 -2.703125 L 2.265625 -2.703125 L 2.265625 -1.28125 C 2.265625 -1.15625 2.265625 -0.8125 2.609375 -0.8125 C 2.953125 -0.8125 2.953125 -1.15625 2.953125 -1.28125 L 2.953125 -2.703125 L 4.375 -2.703125 C 4.5 -2.703125 4.84375 -2.703125 4.84375 -3.046875 Z M 4.84375 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-2\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.546875 -2.140625 C 4.546875 -2.296875 4.421875 -2.390625 4.40625 -2.40625 C 4.15625 -2.578125 3.484375 -2.90625 3.21875 -3.046875 L 4.296875 -3.625 C 4.421875 -3.703125 4.546875 -3.765625 4.546875 -3.953125 C 4.546875 -4 4.546875 -4.28125 4.140625 -4.28125 L 2.90625 -3.5625 C 2.9375 -3.828125 2.9375 -4.484375 2.9375 -4.78125 C 2.9375 -4.859375 2.9375 -5.1875 2.609375 -5.1875 C 2.28125 -5.1875 2.28125 -4.859375 2.28125 -4.78125 C 2.28125 -4.484375 2.296875 -3.828125 2.3125 -3.5625 L 1.234375 -4.203125 C 1.09375 -4.28125 1.078125 -4.28125 1 -4.28125 C 0.8125 -4.28125 0.671875 -4.109375 0.671875 -3.953125 C 0.671875 -3.765625 0.78125 -3.703125 0.921875 -3.640625 L 2 -3.046875 L 0.921875 -2.46875 C 0.8125 -2.390625 0.671875 -2.328125 0.671875 -2.140625 C 0.671875 -2.09375 0.671875 -1.796875 1.078125 -1.796875 L 2.3125 -2.515625 C 2.296875 -2.265625 2.28125 -1.609375 2.28125 -1.3125 C 2.28125 -1.21875 2.28125 -0.890625 2.609375 -0.890625 C 2.9375 -0.890625 2.9375 -1.21875 2.9375 -1.3125 C 2.9375 -1.609375 2.9375 -2.265625 2.90625 -2.515625 C 3.515625 -2.1875 3.15625 -2.375 3.84375 -1.96875 C 4.109375 -1.796875 4.140625 -1.796875 4.21875 -1.796875 C 4.421875 -1.796875 4.546875 -1.984375 4.546875 -2.140625 Z M 4.546875 -2.140625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-3\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.9375 -0.296875 C 4.9375 -0.609375 4.6875 -0.609375 4.53125 -0.609375 L 4.140625 -0.609375 L 2.859375 -2.21875 L 4 -3.6875 L 4.390625 -3.6875 C 4.53125 -3.6875 4.8125 -3.6875 4.8125 -3.984375 C 4.8125 -4.296875 4.546875 -4.296875 4.390625 -4.296875 L 3.234375 -4.296875 C 3.078125 -4.296875 2.828125 -4.296875 2.828125 -4 C 2.828125 -3.6875 3.046875 -3.6875 3.3125 -3.6875 L 2.578125 -2.6875 L 1.828125 -3.6875 C 2.078125 -3.6875 2.296875 -3.6875 2.296875 -4 C 2.296875 -4.296875 2.046875 -4.296875 1.90625 -4.296875 L 0.734375 -4.296875 C 0.59375 -4.296875 0.328125 -4.296875 0.328125 -3.984375 C 0.328125 -3.6875 0.59375 -3.6875 0.734375 -3.6875 L 1.140625 -3.6875 L 2.3125 -2.21875 L 1.078125 -0.609375 L 0.671875 -0.609375 C 0.53125 -0.609375 0.265625 -0.609375 0.265625 -0.296875 C 0.265625 0 0.53125 0 0.671875 0 L 1.84375 0 C 2 0 2.25 0 2.25 -0.296875 C 2.25 -0.609375 2.03125 -0.609375 1.71875 -0.609375 L 2.578125 -1.828125 L 3.46875 -0.609375 C 3.1875 -0.609375 2.96875 -0.609375 2.96875 -0.296875 C 2.96875 0 3.21875 0 3.375 0 L 4.53125 0 C 4.671875 0 4.9375 0 4.9375 -0.296875 Z M 4.9375 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-4\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.671875 -3.046875 C 4.671875 -3.40625 4.3125 -3.40625 4.203125 -3.40625 L 1.03125 -3.40625 C 0.90625 -3.40625 0.5625 -3.40625 0.5625 -3.046875 C 0.5625 -2.703125 0.90625 -2.703125 1.03125 -2.703125 L 4.203125 -2.703125 C 4.3125 -2.703125 4.671875 -2.703125 4.671875 -3.046875 Z M 4.671875 -3.046875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-5\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.40625 -0.296875 C 4.40625 -0.609375 4.171875 -0.609375 4 -0.609375 L 3.09375 -0.609375 L 3.09375 -5.796875 C 3.09375 -5.953125 3.09375 -6.203125 2.796875 -6.203125 C 2.609375 -6.203125 2.546875 -6.078125 2.5 -5.96875 C 2.125 -5.109375 1.609375 -5 1.421875 -4.984375 C 1.25 -4.96875 1.046875 -4.953125 1.046875 -4.671875 C 1.046875 -4.421875 1.21875 -4.375 1.375 -4.375 C 1.5625 -4.375 1.96875 -4.4375 2.40625 -4.8125 L 2.40625 -0.609375 L 1.5 -0.609375 C 1.34375 -0.609375 1.109375 -0.609375 1.109375 -0.296875 C 1.109375 0 1.359375 0 1.5 0 L 4 0 C 4.15625 0 4.40625 0 4.40625 -0.296875 Z M 4.40625 -0.296875 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-6\">\n",
       "<path style=\"stroke:none;\" d=\"M 3.234375 -0.625 C 3.234375 -0.984375 2.9375 -1.25 2.625 -1.25 C 2.25 -1.25 2 -0.9375 2 -0.625 C 2 -0.265625 2.296875 0 2.609375 0 C 2.984375 0 3.234375 -0.3125 3.234375 -0.625 Z M 3.234375 -0.625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-7\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.71875 -3.046875 C 4.71875 -4.890625 3.703125 -6.203125 2.609375 -6.203125 C 1.5 -6.203125 0.5 -4.859375 0.5 -3.046875 C 0.5 -1.203125 1.515625 0.109375 2.609375 0.109375 C 3.734375 0.109375 4.71875 -1.21875 4.71875 -3.046875 Z M 4.03125 -3.15625 C 4.03125 -1.671875 3.390625 -0.5 2.609375 -0.5 C 1.828125 -0.5 1.1875 -1.671875 1.1875 -3.15625 C 1.1875 -4.609375 1.875 -5.59375 2.609375 -5.59375 C 3.34375 -5.59375 4.03125 -4.609375 4.03125 -3.15625 Z M 4.03125 -3.15625 \"/>\n",
       "</symbol>\n",
       "<symbol overflow=\"visible\" id=\"glyph-1514921372656250-0-8\">\n",
       "<path style=\"stroke:none;\" d=\"M 4.703125 -1.890625 C 4.703125 -2.90625 3.9375 -3.875 2.765625 -3.875 C 2.40625 -3.875 1.96875 -3.8125 1.5625 -3.59375 L 1.5625 -5.484375 L 3.953125 -5.484375 C 4.109375 -5.484375 4.359375 -5.484375 4.359375 -5.78125 C 4.359375 -6.09375 4.109375 -6.09375 3.953125 -6.09375 L 1.28125 -6.09375 C 0.953125 -6.09375 0.875 -6 0.875 -5.6875 L 0.875 -3.03125 C 0.875 -2.84375 0.875 -2.625 1.203125 -2.625 C 1.375 -2.625 1.421875 -2.671875 1.5 -2.765625 C 1.765625 -3.09375 2.171875 -3.265625 2.75 -3.265625 C 3.5625 -3.265625 4.015625 -2.546875 4.015625 -1.890625 C 4.015625 -1.09375 3.3125 -0.5 2.46875 -0.5 C 2.1875 -0.5 1.5625 -0.578125 1.28125 -1.125 C 1.328125 -1.171875 1.40625 -1.25 1.40625 -1.453125 C 1.40625 -1.734375 1.171875 -1.890625 0.96875 -1.890625 C 0.8125 -1.890625 0.515625 -1.796875 0.515625 -1.421875 C 0.515625 -0.59375 1.359375 0.109375 2.46875 0.109375 C 3.71875 0.109375 4.703125 -0.78125 4.703125 -1.890625 Z M 4.703125 -1.890625 \"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -3.081656 -6.166812 L -10.909781 -21.823062 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.197769 1.594705 C -1.094685 0.995515 -0.00108944 0.101122 0.297636 0.00155534 C -0.00108405 -0.099774 -1.094631 -0.995973 -1.197683 -1.593422 \" transform=\"matrix(-0.44724,0.89442,0.89442,0.44724,56.94813,27.78778)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 3.082406 -6.166812 L 10.887094 -21.772281 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196282 1.596396 C -1.095358 0.997842 -0.000898986 0.0982794 0.298379 0.000842862 C 0.000842751 -0.098339 -1.09534 -0.994441 -1.194514 -1.592997 \" transform=\"matrix(0.44899,0.898,0.898,-0.44899,78.74418,27.73634)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -20.288687 -34.463687 L -36.530875 -50.701969 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.19551 1.591971 C -1.096048 0.995378 0.00052105 0.097774 0.298825 -0.00164419 C 0.000530893 -0.101092 -1.095949 -0.998804 -1.195352 -1.595407 \" transform=\"matrix(-0.70717,0.7071,0.7071,0.70717,31.32967,56.66877)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -14.1715 -34.463687 L -14.1715 -51.018375 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193788 1.59564 C -1.096131 0.997984 0.001525 0.0995463 0.2984 0.00189 C 0.001525 -0.0996725 -1.096131 -0.99811 -1.193788 -1.595766 \" transform=\"matrix(0,1,1,0,53.68561,56.98285)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -16.780875 -61.912906 L -25.081656 -78.514469 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195506 1.592342 C -1.098086 0.997256 -0.000173027 0.097632 0.300845 0.000180819 C -0.00192004 -0.0989889 -1.096418 -0.998534 -1.193879 -1.595354 \" transform=\"matrix(-0.44896,0.898,0.898,0.44896,42.77553,84.47976)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -11.566031 -61.912906 L -2.816031 -79.412906 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.193204 1.59467 C -1.095381 0.995471 -0.0018081 0.0992835 0.298664 0.00145246 C -0.0000629287 -0.0981201 -1.095399 -0.996035 -1.193232 -1.593485 \" transform=\"matrix(0.44722,0.89442,0.89442,-0.44722,65.04482,85.37805)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -34.460562 -91.155094 L -49.694937 -106.389469 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.1976 1.594312 C -1.095386 0.994942 -0.00157523 0.100058 0.296733 0.000631519 C -0.0015696 -0.098812 -1.098092 -0.99652 -1.194748 -1.595895 \" transform=\"matrix(-0.70714,0.7071,0.7071,0.70714,18.16251,112.35614)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -28.347281 -91.155094 L -28.347281 -107.260562 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.1969 1.593269 C -1.095338 0.995613 -0.0015875 0.101081 0.299194 -0.00048125 C -0.0015875 -0.0981375 -1.095338 -0.996575 -1.1969 -1.594231 \" transform=\"matrix(0,1,1,0,39.5122,113.22815)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 0.000375 -90.260562 L 0.000375 -107.709781 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.195991 1.594125 C -1.094429 0.996469 -0.00067875 0.0980313 0.300102 0.000375 C -0.00067875 -0.101187 -1.094429 -0.995719 -1.195991 -1.593375 \" transform=\"matrix(0,1,1,0,67.859,113.67646)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 5.219125 -90.260562 L 21.348031 -106.389469 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.194922 1.59575 C -1.098239 0.99637 -0.00166309 0.0986957 0.29941 0.00202567 C -0.00166028 -0.100177 -1.095448 -0.995121 -1.197639 -1.594503 \" transform=\"matrix(0.7071,0.70712,0.70712,-0.7071,89.20873,112.35612)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -2.70275 -118.604312 L -11.01525 -134.651187 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196118 1.594712 C -1.097024 0.99706 -0.00101585 0.100183 0.299924 0.00173459 C 0.00184422 -0.100046 -1.094665 -0.996895 -1.197591 -1.593949 \" transform=\"matrix(-0.46143,0.89082,0.89082,0.46143,56.84388,140.61874)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 2.7035 -118.604312 L 11.016 -134.651187 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.197247 1.594613 C -1.094321 0.997558 0.000397206 0.0972522 0.300268 -0.00107078 C -0.000672001 -0.0995192 -1.09668 -0.996397 -1.195774 -1.594049 \" transform=\"matrix(0.46143,0.89082,0.89082,-0.46143,78.87412,140.61874)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 14.17225 -34.510562 L 14.17225 -49.561344 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.196259 1.59261 C -1.094696 0.994954 -0.00094625 0.100423 0.299835 -0.00114 C -0.00094625 -0.0987962 -1.094696 -0.997234 -1.196259 -1.59489 \" transform=\"matrix(0,1,1,0,82.03239,55.52829)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.3985;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M 20.308969 -34.483219 L 36.527719 -50.701969 \" transform=\"matrix(1,0,0,-1,67.859,5.966)\"/>\n",
       "<path style=\"fill:none;stroke-width:0.31879;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" d=\"M -1.194881 1.595946 C -1.098225 0.996571 -0.00170307 0.0988623 0.299361 0.00218087 C -0.00170869 -0.100008 -1.095519 -0.994892 -1.197733 -1.594261 \" transform=\"matrix(0.70714,0.7071,0.7071,-0.70714,104.38833,56.66877)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-1\" x=\"65.244\" y=\"8.611\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-2\" x=\"51.071\" y=\"36.908\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-3\" x=\"22.724\" y=\"64.806\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-4\" x=\"51.071\" y=\"64.358\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-2\" x=\"36.897\" y=\"93.601\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-5\" x=\"3.321\" y=\"122.395\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-6\" x=\"8.551386\" y=\"122.395\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-7\" x=\"13.781772\" y=\"122.395\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-3\" x=\"36.897\" y=\"121.499\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-4\" x=\"65.244\" y=\"92.704\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-4\" x=\"65.244\" y=\"121.05\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-8\" x=\"45.33\" y=\"150.742\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-6\" x=\"50.560386\" y=\"150.742\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-7\" x=\"55.790772\" y=\"150.742\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-5\" x=\"74.696\" y=\"150.742\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-6\" x=\"79.926386\" y=\"150.742\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-7\" x=\"85.156772\" y=\"150.742\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-8\" x=\"88.36\" y=\"122.395\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-6\" x=\"93.590386\" y=\"122.395\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-7\" x=\"98.820772\" y=\"122.395\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-1\" x=\"79.417\" y=\"36.958\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-5\" x=\"74.187\" y=\"65.703\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-6\" x=\"79.417386\" y=\"65.703\"/>\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-7\" x=\"84.647772\" y=\"65.703\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use xlink:href=\"#glyph-1514921372656250-0-3\" x=\"107.763\" y=\"64.806\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n",
       "\n"
      ],
      "text/plain": [
       "TreeView.LabelledTree({15, 14} directed simple Int64 graph, Any[:+, :*, :x, :-, :*, 1.0, :x, :-, :-, 5.0, 1.0, 5.0, :+, 1.0, :x])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display(results_pipe.tree, grammar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.2",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
